{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Other Things To Notice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Besides what we've seen so far.\n",
    "\n",
    "There are other things to notice if you want to make training a machine learning model easy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How hard can training be?\n",
    "\n",
    "Let's face it, training is very hard, because of how hard it is to debug a machine learning system. `print`ing out all the values in the model? You'll still not understand why something goes wrong because those numbers are meaningless to you. Trial and error? Well, it's super time-consuming to do so, and will make your computer very heated.\n",
    "\n",
    "But fret not! Most training issues can be categorized into the following types:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient issues\n",
    "\n",
    "Gradient issues are the most common ones. It happens when you're models gradients are out of control, not in the desired range. When [gradients are too large or small](./gradient/norm), or when you are unlucky and stuck in a [saddle point](./gradient/saddle)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning rate\n",
    "\n",
    "Learning rate is how much you update your model. When [learning rate is too small or large](./lr/lr), training may get super slow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimizer\n",
    "\n",
    "An [optimizer](./optimizer/optimizer) is responsible for updating the model. If the wrong optimizer is selected, training can be deceptively slow and ineffective."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch size\n",
    "\n",
    "When you have a [too big or small batch](./batch/batch), bad things happen because of probability."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overfitting and underfitting\n",
    "\n",
    "Sometimes when you're model is [too complicated for the task](./data/overfit), the model looks dope in training but is useless when being used on real world data. Or the model is just [overly simple](./data/underfit) for the task, the model doesn't seem to learn anything in training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "Most of the issues are caused because of how machine learning systems' intolerance to big or small numbers, so carefully selecting/tuning a hyperparameter is key to solving many ML issues encountered in training."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
